1. Field of the Invention
The present invention relates to a device and method for analyzing an input image and calculating an image coding parameter for compressing the image.
2. Discussion of the Related Art
Recently, a capacity of a storage medium or transmission time has been reduced by compressing images. The images electronically stored or transmitted come to be of high resolution and full-colored, and thus of a large capacity. Therefore, it is important to raise a ratio of image compression. Hereinafter, image coding and image compression indicate the same meaning.
There are two types of image compression methods: the lossless image compression and the lossy image compression. When the lossless image compression method is used, a compressed image is completely restored to what it was by decompressing. When the lossy image compression method is used, a compression ratio is expected to be higher than that of the lossless image compression method. However, a compressed image cannot be completely restored by decompressing and the image quality is deteriorated.
In general, under the same coding conditions, the compression ratio and the image quality are inversely proportional to each other. In other words, when the compression ratio is low, a small amount of information is lost and the image quality is maintained. However, if the compression ratio is raised, a large amount of information is lost and the image quality is deteriorated.
An image compression apparatus or method can control the image compression ratio and the image quality by changing an image coding parameter.
In compressing an image, it is desired to maintain a predetermined image quality while compressing the image with the compression ratio as high as possible. That is, it is required to control the coding parameter to provide the best tradeoff between the compression ratio and the image quality so that the compressed image is visually lossless in comparison with the image before compression.
In many cases, an input image has local image characteristics different from one another. In some portions, deterioration in image quality is easily observed and in the other portions, deterioration in image quality is hardly observed. Considering the image quality of the input image as a whole, it is necessary to lower the overall compression ratio to reduce deterioration in image quality in the portions where the deterioration is easily observed.
The following method makes it possible to raise the overall compression ratio while maintaining the image quality.
An input image is divided into blocks. It is then determined whether the deterioration in the image quality is easily observed in each of the divided blocks. A coding parameter that lowers the compression ratio is provided to the blocks where the deterioration in the image quality is easily observed. On the other hand, a coding parameter that raises the compression ratio is provided to the blocks where the deterioration of the image quality is not easily observed. Thereby, the compression ratio in the portions of the image where the deterioration in the image quality is rarely observed can be raised. Thus, the overall compression ratio can be raised while the image quality is uniformly maintained.
In this method, the different coding parameter is selected for each of the blocks. Therefore, the selected coding parameter is added to the head of the code of each block as shown in FIG. 14 and is also coded.
FIG. 19 shows a configuration example of an image coding apparatus used in general to which the present invention is to be applied. In the figure, an input image 1901 is divided by an input image dividing circuit 1902. A divided image 1907 is transmitted to an image quality analysis circuit 1903 and a lossy coding circuit 1905. The image quality analysis circuit 1903 analyzes the divided image 1907 and outputs a coding parameter 1904. The lossy coding circuit 1905 performs coding of the divided image 1907 using the coding parameter 1904 and outputs code 1906.
The present invention relates to the image quality analysis circuit 1903 in FIG. 19.
Conventional examples are described more specifically. As a concrete method able to control the image quality or compression ratio by changing the coding parameter, JPEG is described, for example, in "International Standard of Multimedia Coding", Maruzen Publishing Company, pp. 18-43. In a method using Discrete Cosine Transform (DCT) represented by JPEG, a higher compression ratio is available with the same image quality by using a quantization matrix adaptive to the characteristics of input image blocks.
The DCT is briefly explained by reference to FIG. 15.
In the figure, input image information 1501 is divided into rectangular blocks by a blocking circuit 1502. An orthogonal transform circuit 1503 performs an orthogonal transform on each of the divided blocks of the image information and outputs an orthogonal transform coefficient 1504. The orthogonal transform coefficient 1504 is quantized with a predetermined quantization matrix by a quantization circuit 1505. A coding circuit 1506 provides a code to the quantized orthogonal transform coefficient and outputs it as a code 1507.
In the coding method as shown in FIG. 15, the same quantization process is performed on all blocks. Therefore, deterioration in image quality is caused in the blocks where distortion is prone to occur. On the other hand, in the blocks where distortion rarely occurs, visually useless and unnecessary information is also coded.
As aforementioned, there is a method able to raise the compression ratio while maintaining the image quality by performing different quantization processes on image portions where deterioration in image quality is easily observed and on those where deterioration is rarely observed. The method is explained by reference to FIG. 16. Constituents corresponding to constituents in FIG. 16 have the same reference numbers as those in FIG. 15.
In FIG. 16, each of the divided blocks of the image information is transmitted to an image analysis circuit 1508 and characteristics of the image in the block are analyzed. A result of the analysis 1509 is transmitted to a quantization process selection circuit 1510 and a quantization process is selected. In the DCT coding method, specifically, the quantization process selection circuit 1510 selects a quantization matrix optimum to the result of the analysis 1509. The selected quantization process 1511 is transmitted to the quantization circuit 1505 to be performed. Other circuits operate as same as those corresponding thereto shown in FIG. 15.
In the above-described methods, the image analysis method for analyzing an image and calculating a coding parameter is important for improving both image quality and compression ratio. There are many types of analysis methods. Two examples of conventional image analysis device are described as follows by reference to FIG. 16. FIG. 16 shows main parts extracted from the two conventional examples and rearranged.
A conventional example 1 is disclosed by Japanese Patent Application Laid-Open No. 6-165149. It is determined whether input image blocks are suitable to be coded by the coding method employed here. If they are determined to be suitable, high image quality is expected in the result of coding, and accordingly, they are coded with a high compression ratio. Otherwise, if they are not suitable, low image quality is expected. Therefore, the blocks are coded with a low compression ratio to improve the image quality.
In the conventional example 1, the image analysis circuit 1508 shown in FIG. 16 calculates a physical quantity 1509 indicating probability of occurrence of mosquito noise in each block. The quantization process selection circuit 1510 selects a quantization matrix corresponding to the physical quantity 1509 to raise the compression ratio while maintaining the same image quality. Then a code amount in the blocks showing high probability of occurrence of the mosquito noise is controlled to be increased, and a code amount in the blocks showing low probability of occurrence of the mosquito noise is controlled to be decreased. Thus, a higher compression ratio is available with the uniform image quality.
More specifically, the image analysis circuit 1508 of the conventional example 1 applies a 3.times.3 filter window to each of the pixels in a block. An average of absolute values of differences in the gradation level between the center pixel and surrounding pixels is calculated. Then the number of pixels such that a ratio of the average to a range of a gradation level signal in the block is not more than a predetermined threshold value is calculated. If the number of such pixels in a block is not less than a predetermined value, it is determined that the mosquito noise rarely occurs. If the number of pixels is less than the threshold value, it is determined that the mosquito noise is prone to occur.
Two parameters are used here: the threshold value for the ratio of the average of the absolute values of differences in the gradation level between the pixels to the range of the gradation level signal in a block; and the threshold value for the number of pixels such that their ratio of the average of the absolute values of the differences to a range of a gradation level signal in the block is not more than a predetermined value.
A conventional example 2 is disclosed by U.S. Pat. No. 5,121,216. In this example, when distortion is generated in an input image block by coding, it is determined whether the distortion is visually observed with ease. If it is determined to be easily observed, coding is performed with a low compression ratio. If it is determined otherwise, a high compression ratio is used. Thereby, a higher compression ratio is available with the image quality for which the human visual system is not sensitive to deterioration.
In the conventional example 2, a higher compression ratio is adopted for a complex image because distortion is rarely observed in the complex image. Specifically, the DCT is performed in the input blocks. Then a quantization characteristic is determined corresponding to an absolute value of the (K+1)th largest DCT coefficient. If the (K+1)th DCT coefficient becomes larger, a quantization step is also made larger. Since a block having the large (K+1)th DCT coefficient is a complex block, it is expected that the distortion is rarely observed even if the quantization step is increased. A block having the small (K+1)th DCT coefficient is not a complex block, and therefore the quantization step is reduced so that the distortion is not easily observed.
The parameter used in the conventional example 2 is K.
In the conventional example 1, the quantization step is reduced for an image expected to be prone to show the mosquito noise, and it is increased for an image expected to rarely show the mosquito noise. In the conventional example 2, the quantization step is increased for an image having a large DCT coefficient, and it is reduced for an image having a small DCT coefficient.
In the conventional example 1, if it is correctly analyzed that the input image is really prone to generate the mosquito noise, the above method of controlling the quantization step is suitable. However, in the conventional example 1, it is not assured that the employed analysis method properly corresponds to tendency of occurrence of the mosquito noise. If the analysis method does not properly analyze the tendency of occurrence of the mosquito noise, it is difficult to modify the method to properly correspond thereto because the number of parameters used in the method is only two.
In the conventional example 2, if it is correctly analyzed that the input image is really complex and the distortion is masked, the above method of controlling the quantization step is suitable. However, in the conventional example 2, it is not assured that the employed analysis method properly corresponds to probability of masking the distortion. If the analysis method does not correctly analyze the probability of masking the distortion, it is difficult to modify the method to properly correspond thereto because the number of parameters used in the method is only one.
However, even if the analysis method used in the conventional example 1 correctly analyzes the tendency of occurrence of the mosquito noise, the conventional example 1 cannot be applied in the following case: an image is analyzed and determined to generate the mosquito noise, but the noise is masked and cannot be observed because of the complexity of the image. For example, the conventional example 1 determines the input image block shown in FIG. 17 to be expected to generate a large amount of mosquito noise. However, the mosquito noise actually occurs in an image portion of a fair size around an edge. Therefore, the mosquito noise rarely occurs in the image shown in FIG. 17 because of a small sized portion between the edges. In this case, the distortion is not observed even if the compression ratio is raised. However, the compression ratio cannot be actually raised since the method determines otherwise.
The conventional example 2 cannot be applied to the following case. The input image shown in FIG. 18(a) consists of pixels each having a value of 0 or 255. However, parts of the pixel values are changed in the decoded image as shown in FIG. 18(b). That is, the mosquito noise occurs in the decoded image. If this input image block is DCT transformed, extremely large DCT coefficients are generated for the whole DCT coefficient area as shown in FIG. 18(c). According to the analysis of the DCT coefficient area, it is determined that the image is complex and it may be considered that the quantization step should be increased. However, the quantization step actually cannot be increased because of the mosquito noise. If the quantization step is increased, the mosquito noise occurs and the image quality is deteriorated.
The problems described above are summarized as follows.
(1) In each of the conventional examples, factors affecting the image quality are predicted only based on a predetermined physical quantity (for example, the output of the 3.times.3 filter window in the conventional example 1 and the (K+1)th largest DCT coefficient in the conventional example 2). With such predetermined physical quantity, the number of parameters used for calculating each physical quantity is limited. Therefore, it is difficult to modify the matrix to finally determine the image quality taking the factors that affect the image quality into consideration.
(2) It is considered that there are plural factors that affect the image quality. However, in the conventional examples, only one factor is taken into account or the plural factors cannot be dealt with. Consequently, deterioration in the image quality occurs or the compression ratio is lowered.